Method and system for operating a teleoperated surgical instrument and a manual instrument

ABSTRACT

A system comprises a teleoperated manipulator, a manually operated instrument, and a position sensor coupled to the manually operated instrument. The position sensor is configured to generate pose information for the manually operated instrument. The system further comprises a controller configured to map a pose of the manipulator to a fixed world reference frame based on three-dimensional pose information for the manipulator known internally within the system. The controller is further configured to map a pose of the manually operated instrument to the fixed world reference frame based on the pose information for the manually operated instrument.

RELATED APPLICATION

This application is a divisional of U.S. patent application Ser. No.16/788,045, filed Feb. 11, 2020, which is a divisional of U.S. patentapplication Ser. No. 15/014,559, filed Feb. 3, 2016, now U.S. Pat. No.10,588,703, which is a continuation of U.S. patent application Ser. No.12/839,642, filed Jul. 20, 2010, now U.S. Pat. No. 9,285,246, whichclaims priority to and the benefit of U.S. Provisional Application No.61/304,094, filed Feb. 12, 2010, and entitled “METHOD AND SYSTEM FORABSOLUTE THREE-DIMENSIONAL MEASUREMENTS USING A TWIST-INSENSITIVE SHAPESENSOR,” naming as inventors, Giuseppe M. Prisco, Theodore W. Rogers,Vincent Duindam, Myrian J. Curet, Catherine J. Mohr, Katherine D. Stoy,all of which are incorporated herein by reference in their entireties.

BACKGROUND Field of Invention

Aspects of this invention are related to sensing information tocharacterize a minimally invasive surgical instrument, and moreparticularly to shape sensors used to obtain shape information andabsolute three-dimensional pose information for the minimally invasivesurgical instrument.

Related Art

A recurring problem in minimally invasive surgery is determining thepose and/or shape of an instrument used in the surgery. Also,determining anatomical locations in a patient and correlating MRIimages, or other images to images of the patient taken during surgeryrequires three-dimensional measurements.

Current methods used to obtain shape information about a flexiblesurgical instrument include camera and vision based methods, inertialand gyroscopic sensors based methods, and electromagnetic field sensorbased methods. Also, there are mechanical systems that can touch twodifferent locations and determine the relative relationships between thetwo locations.

These methods are limited in sensing precision and accuracy, sensitivityto environmental conditions, invasiveness in practical application, andcan generally only provide position information on a certain discretenumber of points on the surgical instrument. Also, the distancesmeasured are relative distances and not absolute distances.

Optical fiber shape sensors are known. See e.g., U.S. Pat. No. 5,798,521(filed 27 Feb. 1997), U.S. Pat. No. 6,389,187 B1 (filed 17 Jun. 1998),U.S. Patent Application Pub. No. US 2006/0013523 A1 (filed 13 Jul.2005), and Roger G. Duncan et al., Characterization of a Fiber-OpticShape and Position Sensor, Smart Structures and Materials 2006: SmartSensor Monitoring Systems and Applications (D. Inaudi et al. eds.), 6167Proceedings of SPIE 616704 (16 Mar. 2006), all of which are incorporatedby reference. Optical fiber shape sensors have been used to sense jointangles in a minimally invasive surgical instrument. See, e.g., U.S.Patent Application Pub. No. US 2007/0156019 A1 (filed 20 Jul. 2006) andNASA Tech Briefs, NASA-Inspired Shape-Sensing Fibers Enable MinimallyInvasive Surgery (1 Feb. 2008),http://www.techbriefs.com/content/view/2585/, both of which areincorporated herein by reference.

Typically, optical fiber shape sensors operate via optical time domainreflectometry (OTDR) or via optical frequency domain reflectometry(OFDR). Most approaches use a backscatter method that measures changesin backscattered light caused by a change in the shape of the opticalfiber. Scattering mechanisms that have been used include Rayleighscattering, Raman scattering, Brillouin scattering, and Fluorescencescattering. Fiber Bragg Gratings and the Kerr effect have also been usedin shape-sensing sensors. Some of the Fiber Bragg Grating shape sensorsare insensitive to twist. Consequently, if the shape sensor is twisted,the pose information obtained using the twisted shape sensor is notcorrect.

Other techniques have also been used for shape sensing. For example,electromagnetic fields have been used in combination with sensorsmounted on an endoscope to determine a position of an endoscope. SeeU.S. Patent Application Publication No. 2007/0249901 A1 (filed Mar. 28,2006; disclosing “Instrument Having Radio Frequency IdentificationSystems and Methods for Use.”), which is incorporated herein byreference in its entirety. Also, the changes in resistance of apiezoresistive flexible shape sensor have been used to measure shapechanges.

SUMMARY

In one aspect, an apparatus includes a reference fixture. The referencefixture includes a joint, and, in one aspect, a joint tracker to trackmotion of the joint. The apparatus also includes a surgical instrument.A tether is connected between the joint and the surgical instrument. Ashape sensor extends from the reference fixture through the joint,through the tether, and into the surgical instrument. The shape sensoris substantially free of twist.

Information from the shape sensor in combination with information fromthe joint tracker provides absolute three-dimensional informationrelative to the reference fixture, i.e., provides absolutethree-dimensional information in a fixed world reference frame. Thisabsolute three-dimensional information in a fixed world reference framecontrasts with information from conventional methods that providerelative distances between elements of a surgical instrument, relativedistances between surgical instruments, or a relative distance betweenlocations in a patient undergoing the medical procedure.

The ability to obtain absolute three-dimensional shape and pose data ina fixed world reference frame provides a new level of capability inmedical procedures. In addition, the absolute three-dimensional posedata (position and orientation) in the world fixed reference framepermits registering various images, for example, in the world fixedreference frame and superimposing the images as needed to facilitate amore informed surgical process.

In one aspect, the joint has only one degree of freedom. For example,the joint is a roll joint and the one degree of freedom is roll. Inanother example, the joint includes a ball and screw joint. In anotheraspect, the joint has multiple degrees of freedom that include the onedegree of freedom.

The number of degrees of freedom in the joint or combination of jointsis selected to facilitate moving the surgical instrument by a surgeon.The torsionally stiff tether between the surgical instrument and thereference structure may inhibit manipulation of the surgical instrumentby the surgeon. Thus, a joint or a combination of joints is used in thereference structure to facilitate manipulation of the surgicalinstrument thru all degrees of freedom of the surgical instrument. Inaspects where manipulation of the surgical instrument is not inhibitedby the tether, or the inhibition is not significant in use of thesurgical instrument, a joint with zero degrees of freedom may be used inthe reference structure and the joint tracker is not needed.

The tether defines a lumen extending along a longitudinal axis of thetether, and the shape sensor extends through the lumen. In one aspect,the tether is torsionally stiff.

In another aspect, a liner extends through the lumen in the tether. Theliner includes a liner lumen and the shape sensor extends through theliner lumen. The liner is made from a material such that the shapesensor rotates freely within the liner. In one aspect the material is afluorinated polymer.

In another aspect, a processor processes information, from a jointtracker connected to a reference fixture, to generate a first pose. Thejoint tracker measures motion of a joint connected to the tether. Thetether couples the joint to a surgical instrument. The reference fixtureis affixed to a location to define a fixed world reference frame forminimally invasive surgery.

The processor also processes information from a shape sensor extendingthrough the joint and the tether to generate second pose information.The processor uses the first pose information and the second poseinformation to generate an absolute three-dimensional pose of a proximalpart of the surgical instrument in the fixed world reference frame.

In one aspect, the shape sensor extends to a distal portion of thesurgical instrument. The processor processes information from thissegment of the shape sensor to generate third pose information. Theprocessor uses the first pose information, the second pose informationand the third pose information to generate an absolute three-dimensionalpose of the distal portion of the surgical instrument in the fixed worldreference frame. The processor also generates a shape of the surgicalinstrument in the fixed world reference frame.

In one aspect, the surgical instrument is a flexible surgical instrumentsuch as an endoscope. In another aspect, the surgical instrument is acannula. In yet another aspect, the surgical instrument is a snakelikerobotic surgical instrument, which is one example of a flexible surgicalinstrument.

In another aspect, a method couples a surgical instrument to a distalend of a tether. A proximal end of the tether is coupled to a joint of areference structure. The reference structure also includes a jointtracker that measures motion of the joint. This method extends a shapesensor through the joint and the tether into the surgical instrument.The shape sensor provides information for determining an absolutethree-dimensional pose of a part of the surgical instrument relative tothe reference structure.

In one aspect, the surgical instrument in this method is a cannula andthe method also includes connecting the reference structure to an armsupporting a manipulator of a minimally invasive teleoperated surgicalsystem.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagrammatic view of a portion of a minimally invasivesurgical system, which includes elements to generate absolutethree-dimensional pose and shape data, in a fixed world reference frame,characterizing a surgical instrument.

FIG. 2A is a cross sectional view of a tether and a shape sensor havinga first non-circular cross section.

FIG. 23 is a cross sectional view of a tether and a shape sensor havinga second non-circular cross section.

FIG. 3 is a perspective cross sectional view of one aspect of areference fixture.

FIG. 4A is a diagrammatic view of a portion of a minimally invasiveteleoperated surgical system, which includes elements to generateabsolute three-dimensional pose, in a fixed world reference frame, for aplurality of surgical instruments.

FIG. 4B is a diagrammatic view of a portion of a minimally invasiveteleoperated surgical system, which includes elements to generateabsolute three-dimensional pose and shape data, in a fixed worldreference frame, for another plurality of surgical instruments.

FIG. 5 is a diagrammatic view of an optic fiber core.

FIG. 6 is a diagrammatic view of the proximal end of an optic fiber withan illustrative reference frame.

FIG. 7 is a diagrammatic view that illustrates reference frames atsegment start locations in an optical fiber used for shape sensing.

In the drawings, the first digit of a reference number indicates thefigure in which the element with that reference number first appeared.

DETAILED DESCRIPTION

Aspects of this invention include an apparatus 100 that in turn includesa combination of (1) a reference fixture 115 (FIG. 1) including a joint110, and a joint tracker 111 to track motion of joint 110, and (ii) ashape sensor 140 to provide pose information of surgical instrument 130.Using information from joint tracker 111 and information from shapesensor 140, a processor module 182 generates an absolutethree-dimensional shape of surgical instrument 130, and an absolutethree-dimensional pose of surgical instrument 130 in a fixed worldreference frame at location Lref.

As explained more completely below, location Lref for the fixed worldreference frame can be fixed for a particular procedure at a locationdesired by the surgeon. For example, the fixed world reference frame atlocation Lref is fixed with respect to a patient undergoing surgery, isfixed with respect to a manipulator of a minimally invasive teleoperatedsurgical system, etc. In some procedures, location LO and location Lrefcould be the same location.

Typically, reference fixture 115 is mounted at a location LO. Therelationship between a pose of location LO and the fixed world referenceframe is known. Thus, controller 180 maps the pose of location LO tofixed world reference frame at location Lref.

In one aspect, after reference fixture 115 is fixed in position atlocation LO, location Lref is assigned to another location. For example,the fixed world reference frame at Lref is chosen to be at the mouth ofpatient 190 with x-axis and z-axis in the sagital plane of the patient,z-axis upward perpendicular to operating table 191, and x-axis aimedtoward the feet of the patient. Then, in a registration step, a tip ofsurgical instrument 130 is used to touch several predetermined referencepoints on patient 190 and operating table 191. The processor uses themeasured positions of these reference points relative to referencefixture 115 to compute the position and orientation of the frame atlocation LO relative to the fixed world reference frame Lref.

In this example, location Lref is at the mouth of the patient and thepose of location LO, which is the start of shape sending, is mapped intofixed world reference frame at location Lref. Thus, the absolutemeasurements described more completely below with respect to thereference frame at a location on reference fixture 115 are mapped to thefixed world reference frame at location Lref. Absolute measurements inthe reference frame located at location LO in fixture 115, as describedmore completely below, are automatically mapped to absolute measurementsin the fixed world reference frame at location Lref on the patient, inthis example.

When the fixed world reference frame at location Lref is fixed withrespect to the patient undergoing surgery, apparatus 100 providesabsolute three-dimensional shape and pose data for surgical instrument130 with respect to the patient. This absolute information contrastswith information from conventional methods that provide relativedistances between elements of a surgical instrument, relative distancesbetween surgical instruments, or a relative distance between locationsin a patient undergoing the medical procedure.

The ability to obtain absolute three-dimensional shape and pose data ina fixed world reference frame provides a new level of capability inmedical procedures. In addition, the absolute three-dimensional posedata (position and orientation) relative to the world fixed referenceframe permits registering various images, for example, in the worldfixed reference frame and superimposing the images as needed tofacilitate a more informed surgical process.

In the aspect of FIG. 1, reference fixture 115 includes a joint 110having at least one degree of freedom and a joint tracker 111. Referencefixture 115 is adapted to be affixed to a location of interest, e.g., apart of an operating table, a location on a minimally invasiveteleoperated surgical system, etc. The means used to affix referencefixture 115 is any conventional means that securely attaches referencefixture to the desired location during use.

A tether 120 is connected between joint 110 of reference fixture 115 andsurgical instrument 130. In one aspect, tether 120 has a proximal endconnected to joint 110 in reference fixture 115. A distal end of tether120 is connected to surgical instrument 130. Motion of the proximal endof tether 120 in the one degree of freedom results in motion of joint110 in the one degree of freedom. Properties of tether 120 are describedmore completely below. The movement of joint 110 assists the surgeon inmanipulating surgical instrument in the one degree of freedom, becausejoint 120 facilitates movement of tether 120 in that degree of freedom.

Motion of the proximal end of tether 120 is transferred to joint 110.Joint tracker 111 measures the motion of joint 110 and provides theinformation to controller 180. In this example, a joint with a singledegree of freedom is considered, and in the example described morecompletely below, the joint is a roll joint.

The use of a joint with a single degree of freedom is illustrative onlyand is not intended to be limiting. As described more completely below,one or more degrees of freedom can be measured at the reference fixture.The use of a roll joint is also illustrative only. Another joint thatcould be used is a ball and screw joint. In view of this disclosure,those knowledgeable in the field can implement a joint or a combinationof joints at the reference fixture, which have an appropriate number ofdegrees of freedom to facilitate the use of the surgical instrumentwhile coupled to a tether.

Surgical instrument 130 can be any one of a number of different surgicalinstruments. For example, surgical instrument can be an instrument usedin laparoscopic procedures, an instrument used in Natural OrificeTransluminal Endoscopic Surgery (NOTES) procedures, a cannula, asnake-like robotic surgical instrument, a probe used for establishingreference positions, or any flexible medical instrument used in amedical procedure for which absolute three-dimensional pose and shapeinformation for that flexible medical instrument is useful.

In the aspect of FIG. 1, surgical instrument 130 is represented as aflexible surgical instrument having an end effector at the distal end.This is illustrative only and is not intended to limit surgicalinstrument 130 to only flexible surgical instruments.

A shape sensor 140 extends from reference fixture 115, thru joint 110and joint tracker 111, through tether 120, and into surgical instrument130 to a distal portion of surgical instrument 130. In the aspect ofFIG. 1, shape sensor 140 is illustrated as passing through a lumen ineach of reference fixture 115, joint 110 and joint tracker 111, tether120, and surgical instrument 130. This is illustrative only and is notintended to be limiting to this specific example. In view of thisdisclosure, those knowledgeable in the field can position shape sensor140 relative to reference fixture 115, thru joint 110 and joint tracker111, through tether 120 and surgical instrument 130 to provide thenecessary information to determine the absolute three-dimensional poseand shape of the surgical instrument being used.

In one example, shape sensor 140 is a shape-sensing optic fiber andOptical Frequency Domain Reflectometry is utilized in interrogator 170.In particular, a shape-sensing optic fiber utilizes Fiber BraggGratings. The use of Fiber Bragg Gratings is illustrative only and isnot intended to be limiting to this specific mechanism for creatingbackscatter. In view of this disclosure, other optic fiber shape-sensingtechnologies could be used including but not limited to Rayleighscattering, Raman scattering, and Brillouin scattering.

Further, the use of a shape-sensing optic fiber is also illustrative andis not intended to be limiting. For a particular type of shape sensor140, an interrogator for that shape sensor is used. The data from theinterrogator is processed using techniques equivalent to those describedmore completely below to obtain the absolute three-dimensional data.Accordingly, as stated previously, this example is illustrative of usinga shape-sensing optic fiber to generate absolute three-dimensional poseinformation and is not intended to be limiting to the specificshape-sensing optic fiber described.

In one aspect, shape sensor 140 is implemented using a multi-coreshape-sensing optic fiber. In one example, the optic fiber is a threecore optic fiber, as described more completely below. In otherembodiments, various numbers of cores may be used.

Interrogator 170 interrogates the optic fiber in shape sensor 140 andprovides shape information to an input/output module 181 in controller180. A processor in processor module 182 processes the informationreceived by input/output module 181 using a shape information processingmodule 184.

Using information from joint tracker 111, the pose of the referenceframe at location LO is determined. Note that, in this example, theposition of the reference frame at location LO is fixed, but theorientation of the reference frame at location LO can change as joint110 moves. Depending on the type of joint 110, different degrees offreedom in position and orientation of the reference frame at locationLO change as the joint moves. The processor performs an integration, asdescribed more completely below, using the information from shape sensor140. The integration generates pose information for a frame at locationLi in the reference frame at location LO. The processor combines thispose information with the pose of the reference frame at location LO togenerate pose information for the frame at location LI in the fixedworld reference frame at location Lref.

Similarly, integration of information from shape sensor 140 fromlocation LO to location L2 generates pose information for a frame atlocation L2, in the reference frame at location LO. Combining this withthe pose of reference frame at location LO generates pose informationfor the frame at location L2 in the fixed world reference frame atlocation Lref. This integration also provides shape information ofsurgical instrument 130 between locations Li and L2 in fixed worldreference frame at location Lref.

In this example, the initial position (x₀, y₀, z₀) at location LO isfixed and known and is mapped to the world fixed reference frame atlocation Lref. This initial position in the world fixed reference frameis used in the example described more completely below in offset vectorv ₀ (See expression (17)). The orientation of the fixed world referenceframe including the measurement from joint tracker 111 is used incoordinate system matrix C ₀, (See expression (9)).

A more detailed description of one process used to obtain coordinates(x_(n), y_(n), z_(n)) at curvilinear coordinate L0+81, i.e. at locationL1, is described in copending and commonly assigned U.S. PatentApplication Publication No. US 2009/0324161 A1, of U.S. Patentapplication Ser. No. 12/64,829, entitled “FIBER OPTIC SHAPE SENSOR,” ofGiuseppe M. Prisco, and filed on 30 Jun. 2008, which is incorporatedherein by reference in its entirety.

Shape sensor 140 is fixed in position proximal to joint 110 and jointtracker 111. (Here, distal is away from joint 110 and joint tracker 111towards surgical instrument 130 and proximal is the opposite direction.)Location LO is typically at a center line of joint 100 in joint tracker111 so that shape sensor 140 is fixed in position at a location removedfrom location LO, e.g., removed from the region being sensed.

Techniques and methods for fixing a shape-sensing optic fiber inposition are described in commonly assigned U.S. patent application Ser.No. 12/164,297, entitled “FIXTURE FOR SHAPE-SENSING OPTICAL FIBER IN AKINEMATIC CHAIN,” naming as inventors Theodore W. Rogers and GiuseppePrisco, and filed on 30 Jun. 2008, which is incorporated herein byreference in its entirety. Also, conventional techniques for creating aservice loop in an optic fiber that are used in the telecommunicationsindustry could be used.

In one aspect, shape sensor 140 has a known feature, such as a knowngeometrical shape, at location L1 where shape sensor 140 enters surgicalinstrument 130. In one aspect, the distal end of shape sensor 140 insurgical instrument 130 is free to move.

Suitable techniques for implementing a known feature in shape sensor 140at the proximal entry location in surgical instrument 130 are describedin commonly assigned U.S. patent application Ser. No. 12/618,000,entitled “FIBER OPTIC SHAPE SENSOR,” naming as inventor, Giuseppe M.Prisco, and filed on 13 Nov. 2009, which is incorporated herein byreference in its entirety. See also, commonly assigned U.S. patentapplication Ser. No. 12/618,082, entitled “Method and System to SenseRelative Partial-Pose Information Using a Shape Sensor,” naming asinventors, Vincent Duindam et al., filed on 13 Nov. 2009, which isincorporated herein by reference in its entirety.

In one aspect, the known feature is an S-curve, in shape sensor 140 witha geometrical center at location L1. The lumen in which shape sensor 140is located can be shaped to provide the known feature, or alternatively,a clamp or clamps on shape sensor 140 can be used to create the knownfeature.

The use of a known feature at the entry point of shape sensor 140 isoptional. Alternatively, the length of the tether and the length of anydistance in reference fixture 115 from location LO to the proximal endof the tether can be measured to determine location L1. In someapplications, location L1 is not of interest. Typically, for a flexiblesurgical instrument, the shape of the last half meter to meter of thedistal end of the surgical instrument and the pose of the distal end ofthe surgical instrument are of interest.

In the aspect where shape sensor 140 is a multi-core shape-sensing opticfiber, care is taken to prevent shape sensor 140 from twisting in tether120 and surgical instrument 130, because twisting distorts the measuredshape information. In fact, in some conventional systems that reliedonly on a shape sensor to determine three-dimensional poses, thepresence of twist resulted in inaccurate determinations.

In one aspect, twisting of shape sensor 140 (rotation about itstangential axis) is mitigated by using a torsionally stiff material forthe body of tether 120 and for the body of surgical instrument 130.Suitable materials for the bodies include material used in endoscopebodies.

Herein, a torsionally stiff body is a body that has a twist smaller thana threshold while used in normal loading and normal use situations. Thethreshold is the amount of twist that would bother a surgeon while usingsurgical instrument 130 because the visual image of instrument 130 didnot follow the motions made by the surgeon in manipulating surgicalinstrument 130 due to measurement error introduced by twisting of theshape sensor. Experiments have shown that a threshold of plus or minusten degrees of twist is acceptable to most surgeons.

To further limit any twist of shape sensor 140, a torsionally stiff bodycan be used in combination with a lumen in the body having anon-circular cross section and a shape sensor having a same non-circularcross section. Such cross sections prevent the shape sensor fromtwisting within the lumen and so any twist is that imparted by thetorsionally stiff body.

For example, in FIG. 2A, torsionally stiff body 210A has a lumen with anon-circular cross section 211A, and shape sensor 240A has an equivalentnon-circular cross section. In FIG. 2B, torsionally stiff body 210B hasa lumen with a rounded triangular cross section 211B, and shape sensor240B has an equivalent rounded triangular cross section.

The cross sections in FIGS. 2A and 28 are illustrative only and are notintended to limiting. When the cross sectional shape of the shape sensoris known, a lumen with substantially the same cross sectional shape canbe formed in the torsionally stiff tether. For example if the crosssection of the shape sensor is a letter D shape, the cross section ofthe lumen in the tether would have a letter D shape.

In another aspect, twisting of shape sensor 140 is mitigated byenclosing shape sensor 140 in low-friction tubing so that shape sensor140 rotates freely within the lumen of the tubing and does not twistbeyond the threshold irrespective of any twist in the tubing. Thelow-friction tubing is passed through the lumens in the tether and inthe surgical instrument.

As an example, the low-friction tubing is made from a fluorinatedpolymer. One example of a fluorinated polymer is polytetrafluoroethylene(PTFE).

Alternatively, a torsionally stiff body can be used in combination withlow-friction tubing inserted in the lumen of the torsionally stiff body.Irrespective of whether a torsionally stiff tether body is used,low-friction tubing is used in a lumen of the tether, or a combinationof the two is used, the shape sensor remains substantially twist free.Herein, substantially twist free means that the twist of the shapesensor is less than the threshold described above for the torsionallystiff tether.

FIG. 3 is a perspective cross-sectional view of one example of areference fixture 315. A body 350 of reference fixture 315 has a lumen351 extending through body 350 from a first edge surface 352 to a secondedge surface 353. Body 350 also includes a plurality of threaded screwholes extending into body 350 from first edge surface 352, and fromsecond edge surface 353. In FIG. 3, threaded screw holes 363-1, 363-3,364-1, 364-3 in the plurality of threaded screw holes are visible. Theplurality of screw holes is used to attach roll joint 310 to body 350.

Roll joint 310 includes a first bearing 361 and a second bearing 362mounted in lumen 351. First and second bearings 361, 362 support ahollow rod 360 that passes through lumen 351. Hollow rod 360 extendsbeyond first edge surface 352.

Tether 320 is connected to the end of hollow rod 360 extending beyondfirst edge surface 352 by a clamp 321. Thus, roll motion of the proximalend of tether 320 is transferred to rod 360.

Hollow rod 360 also extends beyond second edge surface 352 and passesthrough a lumen in roll joint tracker 311 that is affixed to hollow rod360. Thus, as rod 360 turns, the roll motion is transferred to rolljoint tracker 311 and is measured by roll joint tracker 311. Roll jointtracker 311 is a conventional optical joint tracker and so is known tothose knowledgeable in the field.

Use of an optical joint tracker is illustrative only and is not intendedto be limiting. Upon selection of a joint or joints, appropriatetracking mechanisms can be selected. If the joint has no degrees offreedom, the joint tracker is not used.

Shape sensor 340 passes through hollow rod 360 and is fixed in positionafter shape sensor 340 passes out of the end of rod 360 adjacent to rolljoint tracker 311. Shape sensor 340 is fixed in position as describedabove.

In the example of FIG. 3, a single degree of freedom is restrained byfixture 315. In other implementations, additional degrees of freedom canbe restrained by the reference fixture to improve the manipulability ofthe surgical instrument.

The ability to obtain three-dimensional shape and pose measurements in afixed world reference frame enhances the capabilities available forminimally invasive surgical procedures. Two examples are considered. Ina first example, a minimally invasive teleoperated surgical system and alaparoscopic system are used in combination. In a second example, only alaparoscopic system is considered.

The first example includes a conventional minimally invasiveteleoperated surgical system, such as the da Vinci' Surgical System,manufactured by Intuitive Surgical, Inc., Sunnyvale, Calif., andutilizes internal system knowledge that the system has about itself. Thesecond example is a totally laparoscopic version which gathers allinformation directly with a laparoscopic positioning system.

In the first example with the minimally invasive robotic surgicalsystem, only a portion of the system is illustrated in FIG. 4A. A cart405 supports a plurality of manipulators. In this example, an endoscopiccamera manipulator 406 is illustrated.

A reference fixture 415, mounted on endoscopic camera manipulator 406,is similar to reference fixture 115 except reference fixture 415 isconnected to a plurality of tethers, which in this example includestether 421 and tether 422. Again, in this example, the fixed worldreference frame at location Lref is defined at a location on patient490. A reference frame for the shape sensor passing through tether 421is defined at location LO-1 and a reference frame for the shape sensorpassing through tether 422 is defined at location LO-2.

In a manner similar to that described above with respect to FIG. 1, thepose in the reference frame at location LO-1 is mapped by controller 180into the fixed world reference frame at location Lref. Similarly, thepose in the reference frame at location LO-2 is mapped by controller 180into the fixed world reference frame at location Lref.

In this example, the shape sensor passing through tether 421 has a knownfeature, such as an S-curve, after leaving tether 421 and enteringcannula 431. Similarly, the shape sensor passing through tether 423 hasa known feature after leaving tether 422 and entering cannula 432.Cannulas 431, 432 are examples of a particular surgical instrument.

In this example, reference fixture 415 includes a first joint trackerthat tracks motion of the joint connected to tether 421 that in turn isconnected to cannula 431, and a second joint tracker that tracks motionof the joint connected to tether 422 that in turn is connected tocannula 432.

However, in some aspects, even though the tether is torsionally stiff,manipulability of the surgical instrument may be adequate withoutproviding a joint with any degrees of freedom in the referencestructure. For example, a cannula may be sufficiently movable whenattached to a torsionally stiff tether that a one-degree of motion jointis not needed. In this situation, the joint in the reference structurehas zero degrees of freedom and the joint tracker is not used.

The known feature of the shape sensor in cannula 431 providesinformation that is processed to determine the pose of cannula 431, atlocation L3. The integration described more completely below ofinformation from shape sensor 140 is performed from location LO-1 tolocation L3 to generate pose information for a frame at location L3, inthe reference frame at location LO-1. Combining this with the pose ofthe reference frame at location LO-1 generates pose information for theframe at location L3 in the fixed world reference frame at locationLref. The process is the same as that described above with respect toFIG. 1. An equivalent process is used to obtain the three-dimensionalpose of cannula 432, at location L4, in the fixed world reference frameat location Lref.

Three-dimensional pose information for the manipulator on whichreference fixture 415 is mounted is known internally within theminimally-invasive surgical system. Thus, the pose information generatedin the fixed world reference frame for cannulas 431, 432 can be mappedinto the reference frame for the manipulator.

An insertion depth of a manually inserted surgical instrument 433 (FIG.4B) is directly measured by the addition of a second tether 423 betweentethered cannula 432 and surgical instrument 433. The shape sensorextends through location LO-2 through tether 422 to cannula 432 and fromcannula 432 through tether 423 to a known feature in surgical instrument433. In the example of FIG. 4B, the pose at location L5 in the fixedworld reference frame at location Lref and the pose at location L4 inthe fixed world reference frame at location Lref are used to determinethe insertion depth. The distance between the poses for the twolocations diminishes as instrument 433 is inserted.

A system utilizing a minimally invasive teleoperated surgical coupledsystem has a significant advantage in that the reference to the cameracan make use of state variables that are known to the system such ascamera angle, depth of insertion of the camera tip and focus depth. In alaparoscopic system only, all of the variables must be directlymeasured.

Direct measurement of all of cannula positions with tethers back to thesame reference point can establish the vector of the instrument cannulaswith respect to the camera cannula in a manner equivalent to thatdescribed above. The depth of the scope insertion can be measured withan additional tether as described above.

As described above, in one aspect, the shape sensor is implemented usinga multi-core optic fiber with Bragg Fiber Gratings. In one example, theoptic fiber is a three core optic fiber, as described more completelybelow. In other embodiments, various numbers of cores may be used.

In this example, interrogator 170 interrogates the optic fiber andprovides shape information from the optic fiber to an input/outputmodule 181 in controller 180. Interrogator 170 implements OpticalFrequency Domain Refleotometry technology with a standard OpticBackscatter Reflectometer for each core in the optic fiber.

The shape information is analyzed as a function of a discretizedcurvilinear coordinate S(t). A processor in a processor module 182 ofcontroller 180 executes computer instructions in a shape informationprocessing module 184 stored in memory 183. The processor determines thereference frame at location LO. The processor then performs anintegration from the location of the reference frame at location LO tothe geometrical center of the known feature at location L1. In oneaspect, the location of the geometrical center of the known feature isdetermined by finding the center of the shape signature representing theknown feature.

The integration (See expression (15) below) generates the x-y-z positionand orientation of the known feature at location L1, which in turn canbe used to determine the position and angle of the geometrical center ofthe known feature relative to the fixed world reference frame atlocation Lref.

A more detailed description of one process used to obtain coordinates(x_(n), y_(n), z_(n)) at curvilinear coordinate LO+S1 is described incopending and commonly assigned U.S. Patent Application Publication No.US 2009/0324161 A1, which was previously incorporated herein byreference in its entirety.

FIG. 5 is a diagrammatic view of one configuration of an optic fibercore 500. Other configurations are illustrated in copending and commonlyassigned U.S. Patent Application Publication No. US 2009/0324161 A1.

In. FIG. 5, the surrounding cladding and fiber are omitted for clarity.Each of the many vertical lines shown represents individual, adjacentFiber Bragg Gratings. As described below, each core is one core of threeor more cores in a single optic fiber.

As indicated above and shown in FIG. 5, a curvilinear coordinate systemS(t) is defined for the fiber, and hence for core 500 and the othercores (not shown). In some cases, location LO of the origin ofcoordinate system S(t) is defined relative to fixture 115, as describedabove.

Once origin location LO is defined, one or more shape-sensing segmentsare defined between locations along the core. Each defined shape-sensingsegment of a core contains part of one Fiber Bragg Grating, or one fullFiber Bragg Grating, or many adjacent Fiber Bragg Gratings. As shown inFIG. 5, core 500 has Fiber Bragg Gratings defined along its entirelength, with shape sensing segment 518 defined at the distal end of thecore, with no additional segments defined in the Fiber BraggGrating-configured core. Shape-sensing segment 518 starts at location L₉and extends for a distance S₉.

FIG. 6 is a diagrammatic view of the proximal end of an optic fiber 600with an illustrative reference frame defined. As shown in FIG. 6, fiber600 has three Fiber Bragg Grating-configured cores 602 a, 602 b, 602 cwithin a cladding layer 600 a. Each core 602 a, 602 b, 602 c ispositioned at an apex of an equilateral triangle centered in optic fiber600.

As shown in FIG. 6, a Cartesian reference frame is defined for opticfiber 600. One axis of the Cartesian reference frame intersects one ofthe cores (the x-axis is shown intersecting core 602 a as anillustration) and another axis is tangent to the centerline of opticfiber 600 (the z-axis is shown as an illustration). Defining the x-axisto extend through a core provides a rotational reference around thecenterline of optic fiber 600. The definition of the x-axis is arbitraryand can be based on the geometry of the kinematic chain embedding theoptic fiber. For instance, the x-axis could be aligned to one joint axisof the kinematic chain in which the optic fiber is embedded orassociated.

The Cartesian reference frame (x, y, z) shown in FIG. 6 functions as abase frame when defined with an origin coincident with the origin of thecurvilinear coordinate system S(t). When a Cartesian reference frame isdefined with an origin at a segment start location, the Cartesianreference frame functions as a shape-sensing segment reference frame. ACartesian reference frame may be similarly defined at a segment endlocation. Although three cores are shown in FIG. 6, other numbers ofcores may be used (e.g., two opposite cores for planar bend measurement,four cores to measure fiber twist, etc.).

FIG. 7 is a diagrammatic view that illustrates reference frames atsegment starts in an optical fiber used for shape sensing. FIG. 7depicts an optical fiber 700 that is, in one embodiment, configured withthree cores as illustrated in FIG. 6 (four cores or other coreconfigurations may be used). Two shape-sensing segments are defined inoptic fiber 700. A first segment 702 a is defined from curvilinearreference location LO (segment start) to curvilinear reference locationLO+S_(I) (segment end). The second segment 702 b is defined fromcurvilinear reference location L1 (segment start) to curvilinearreference location L1+S₂ (segment end). In accordance with an aspect ofthe invention, a first Cartesian reference frame 704 a is defined atsegment start LO. The z-axis of reference frame 704 a is tangent to thecenterline of optic fiber 700 at segment start LO. The x-axis ofreference frame 704 a runs through one of the cores as illustrativelyshown and described in FIG. 6. Similarly, a second Cartesian referenceframe 704 b is defined at segment start L1, with the z-axis of referenceframe 704 b tangent to the centerline of optic fiber 700 at segmentstart L1. The x-axis of reference frame 704 b runs through the same corethe x-axis of reference frame 704 a.

The base reference frame illustrated in FIG. 6 and the two segment startreference frames illustrated in FIG. 7 are interrelated because allthree have one normal axis (e.g., the x-axis) defined through the samecore (e.g., core 702 a).

The following is an illustration of computations carried out by anelectronic data processing unit, sometime simply call a processor, incontroller 180. Skilled individuals will understand that many hardware,firmware, and software options exist for constructing an electronic dataprocessing unit, and that implementation of necessary computations willbe routine in light of this description.

The expression for the local strain e(J) is written as a function ofdistance along a given fiber core,

ε_(n)=ε(Δdn)  (1)

where Δd is the distance increment per index n The Δd value is set bythe resolution of the OFDR-based interrogator. For instance the localstrain ε(s) as a function of distance along each fiber core is obtainedby making use of an “Optical Backscatter Reflectometer”, a commerciallyavailable product from Luna Innovations Incorporated, Roanoke, Va., foreach core. Such a device is able to output the phase derivative of thereflected light as a function of the distance along the fiber core, asshown in Optical Backscatter Reflectometer User Guide Chaps 5-6, 33-60(Luna Technologies, Inc. 2004) (Document version 1.0 for OBR controlsoftware version 0.42 Beta), which is incorporated herein by reference.Such Phase Derivative information is proportional to the desired localstrain ε(S) in expression (1).

For the pre-set perturbation, the differential strains between the coresare needed. For three cores, the required differential strains are:

Δε_(p,n)=ε_(2,n)−ε_(1,n)  (2a)

Δε_(q,n)=ε_(3,n)−ε_(1,n)  (2b)

where Δε_(p) and Δε_(q) designate the two differential strain arrays.

These differential strains can then be converted into local bends in anortho-normal coordinate system by using a simple linear transformation.

$\begin{matrix}{\begin{bmatrix}\theta_{x,n} \\\theta_{y,n}\end{bmatrix} = {\begin{bmatrix}m_{px} & m_{qx} \\m_{py} & m_{qy}\end{bmatrix}\begin{bmatrix}{\Delta\epsilon}_{p,n} \\{\Delta\epsilon}_{q,n}\end{bmatrix}}} & (3)\end{matrix}$

The m-matrix m is a full description of the multi-core fiber, capturingthe effects of the locations of the cores and the initial rotationalorientation of the fiber in the coordinate system.

Next, these two rotation values are used to create a rotation matrixequal to the product of a first rotation of an angle θ_(x,n) around thex-axis and a second rotation of θ_(y,n) around the y-axis according tothe equations:

$\begin{matrix}{{{\overset{\_}{\overset{\_}{R}}}_{x,n} = \begin{bmatrix}1 & 0 & 0 \\0 & {\cos\;\theta} & {{- \sin}\;\theta} \\0 & {\sin\;\theta} & {\cos\;\theta}\end{bmatrix}}{{\overset{\_}{\overset{\_}{R}}}_{y,n} = \begin{bmatrix}{\cos\;\theta} & 0 & {\sin\;\theta} \\0 & 1 & 0 \\{{- \sin}\;\theta} & 0 & {\cos\;\theta}\end{bmatrix}}{{\overset{\_}{\overset{\_}{R}}}_{n} = {{\overset{\_}{\overset{\_}{R}}}_{x,n}\mspace{14mu}{\overset{\_}{\overset{\_}{R}}}_{y,n}}}} & (4)\end{matrix}$

For a small angle approximation, the above expression simplifies to:

$\begin{matrix}{{\overset{\_}{\overset{\_}{R}}}_{n} = \begin{bmatrix}1 & 0 & \theta_{y,n} \\0 & 1 & {- \theta_{x,n}} \\{- \theta_{y,n}} & \theta_{x,n} & 1\end{bmatrix}} & (5)\end{matrix}$

where, because a first order small angle approximation is used, R _(n)is a valid rotation matrix only if θ_(x)<<1 and θ_(y), <<1.

If sufficiently small spatial increments are used, the above conditionsare not difficult to satisfy. This rotation matrix is then moved intothe coordinate system at the n^(th) position on the fiber. In this way,the calculations are iterated to walk down the length of the fiber,reconstructing the tangent vector, as well as the vectors defining therotational coordinate system, along the way. The iterative equation is,

$\begin{matrix}{\overset{\_}{\overset{\_}{C_{n + 1}}} = \overset{\_}{\overset{\_}{C_{n}R_{n}}}} & (6)\end{matrix}$

Or, when using the small-angle approximation,

$\begin{matrix}{\begin{bmatrix}c_{11} & c_{12} & c_{13} \\c_{21} & c_{22} & c_{23} \\c_{31} & c_{32} & c_{33}\end{bmatrix}_{n + 1} = {\begin{bmatrix}c_{11} & c_{12} & c_{13} \\c_{21} & c_{22} & c_{23} \\c_{31} & c_{32} & c_{33}\end{bmatrix}_{n}\begin{bmatrix}1 & 0 & \theta_{y} \\0 & 1 & {- \theta_{x}} \\{- \theta_{y}} & \theta_{x} & 1\end{bmatrix}}_{n}} & (7)\end{matrix}$

And so, the coordinate system at any location along the array is givenby,

$\begin{matrix}{{\overset{\_}{\overset{\_}{C}}}_{p} = {{{\overset{\_}{\overset{\_}{C}}}_{o}{\overset{\_}{\overset{\_}{R}}}_{o}{\overset{\_}{\overset{\_}{R}}}_{1}{\overset{\_}{\overset{\_}{R}}}_{2}\mspace{14mu}\cdots\mspace{14mu}{\overset{\_}{\overset{\_}{R}}}_{p}} = {{\overset{\_}{\overset{\_}{C}}}_{o}\mspace{14mu}\underset{n \propto 0}{\overset{p}{\Pi}}\mspace{14mu}{\overset{\_}{\overset{\_}{R}}}_{n}}}} & (8)\end{matrix}$

The initial value of this coordinate system matrix,

$\begin{matrix}{{\overset{\_}{\overset{\_}{C}}}_{o} = \begin{bmatrix}c_{11} & c_{12} & c_{13} \\c_{21} & c_{22} & c_{23} \\c_{31} & c_{32} & c_{33}\end{bmatrix}_{0}} & (9)\end{matrix}$

Describes the initial orientation of the fiber in the exteriorcoordinate system. If the fiber is initially aligned along the z-axis,the matrix will be,

$\begin{matrix}{{\overset{\_}{\overset{\_}{C}}}_{o} = \begin{bmatrix}{\sin\mspace{14mu}\beta} & {{- \cos}\mspace{14mu}\beta} & 0 \\{\cos\mspace{14mu}\beta} & {\sin\mspace{14mu}\beta} & 0 \\0 & 0 & 1\end{bmatrix}_{0}} & (10)\end{matrix}$

In the description above, the first two vectors still have one degree offreedom, which is the rotation of the fiber around its axis—the samerotational degree of freedom in the m-matrix above. In manyimplementations, this situation is not generally a problem, because itwill generally be taken care of automatically by the way the fiber isembedded in or associated with the kinematic chain and by calibration.Further, it means that complete generality can be retained even if theinitial matrix is restricted to be,

$\begin{matrix}{{\overset{\_}{\overset{\_}{C}}}_{o} = \begin{bmatrix}1 & 0 & 0 \\0 & 1 & 0 \\0 & 0 & 1\end{bmatrix}_{0}} & (11)\end{matrix}$

The tangent vector t is the last column of the C matrix,

$\begin{matrix}{\overset{\_}{t} = {\overset{\_}{\overset{\_}{C}} \cdot \begin{bmatrix}0 \\0 \\1\end{bmatrix}}} & (12)\end{matrix}$

Accordingly, the tangent vector at any particular point is the productof all of the previous rotation vectors,

$\begin{matrix}{{\overset{\_}{\overset{\_}{t}}}_{p} = {{\overset{\_}{\overset{\_}{C}}}_{o}{\prod\limits_{n = 0}^{p}\;{{\overset{\_}{\overset{\_}{R}}}_{n} \cdot \begin{bmatrix}0 \\0 \\1\end{bmatrix}}}}} & (13)\end{matrix}$

The position at any point along the fiber is the sum of all of theprevious tangent vectors, multiplied by the length of fiber that theyrepresent,

$\begin{matrix}{\begin{bmatrix}x \\y \\z\end{bmatrix}_{q} = {\Delta\; d{\sum\limits_{p = 0}^{q}\;{\overset{\_}{\overset{\_}{t}}}_{p}}}} & (14)\end{matrix}$

Substituting in the expression for the tangent vector gives,

$\begin{matrix}{\begin{bmatrix}x \\y \\z\end{bmatrix}_{q} = {\Delta\; d{\sum\limits_{p = 0}^{q}\;\left\lbrack {\left\{ {{\overset{\_}{\overset{\_}{C}}}_{o}{\prod\limits_{n = 0}^{p}\;{\overset{\_}{\overset{\_}{R}}}_{n}}} \right\} \cdot \begin{bmatrix}0 \\0 \\1\end{bmatrix}} \right\rbrack}}} & (15)\end{matrix}$

For generality, an arbitrary offset vector can be added to place thecalculated coordinates into any arbitrary coordinate system.

$\begin{matrix}{{\begin{bmatrix}x \\y \\z\end{bmatrix}_{q} = {{\Delta\; d{\sum\limits_{p = 0}^{q}\;\left\lbrack {\left\{ {{\overset{\_}{\overset{\_}{C}}}_{o}{\prod\limits_{n = 0}^{p}\;{\overset{\_}{\overset{\_}{R}}}_{n}}} \right\} \cdot \hat{z}} \right\rbrack}} + {\overset{\_}{v}}_{0}}}{{where},}} & (16) \\{{{\overset{\_}{v}}_{0} = \begin{bmatrix}x_{0} \\y_{0} \\z_{0}\end{bmatrix}}{{and},}} & (17) \\{\hat{z} = \begin{bmatrix}0 \\0 \\1\end{bmatrix}} & (18)\end{matrix}$

For the computation of the position and orientation of the frame ofreference at the end of a segment with respect to the frame of referenceat the start of the segment, C _(O) is the identity matrix, and v ₀ is avector of zeros, which represents the frame of reference at the start ofthe segment. Alternatively, the computation can be carried in anotherbase or world frame located, for instance, at the base of the kinematicchain. In this case C _(O) is the 3×3 matrix specifying the orientationof the frame of reference at the start of the segment with respect tothe above-mentioned base frame, and v ₀ is the 3×1 vector specifying theposition of the origin of the frame of reference at the start segmentwith respect to the above-mentioned base frame.

As mentioned above, in some instances the quantity Δd is known from theproperty of the particular interferometer that is used. Alternatively,Δd can be calibrated by laying the segment of fiber in a straight line,for instance with the use of a fixture, and comparing the computedsegment tip position from equation 18 with the known segment physicallength.

The above description and the accompanying drawings that illustrateaspects and embodiments of the present inventions should not be taken aslimiting—the claims define the protected inventions. Various mechanical,compositional, structural, electrical, and operational changes may bemade without departing from the spirit and scope of this description andthe claims. In some instances, well-known circuits, structures, andtechniques have not been shown or described in detail to avoid obscuringthe invention.

As used herein, a Fiber Bragg Grating comprises a series of modulationsof a core's refractive index so as to generate a spatial periodicity inthe refraction index. The spacing may be chosen so that the partialreflections from each index change add coherently for a narrow band ofwavelengths, and therefore reflect only this narrow band of wavelengthswhile passing through a much broader band. During fabrication of theFiber Bragg Gratings, the modulations are spaced by a known distance,thereby causing reflection of a known band of wavelengths. However, whena strain is induced on the fiber core, the spacing of the modulationswill change, depending on the amount of strain in the core.

Further, this description's terminology is not intended to limit theinvention. For example, spatially relative terms—such as “beneath”,“below”, “lower”, “above”, •upper”, “proximal”, “distal”, and thelike—may be used to describe one element's or feature's relationship toanother element or feature as illustrated in the figures. Thesespatially relative terms are intended to encompass different positions(i.e., locations) and orientations (i.e., rotational placements) of thedevice in use or operation in addition to the position and orientationshown in the figures. For example, if the device in the figures isturned over, elements described as “below” or “beneath” other elementsor features would then be “above” or “over” the other elements orfeatures. Thus, the exemplary term “below” can encompass both positionsand orientations of above and below. The device may be otherwiseoriented (rotated 90 degrees or at other orientations) and the spatiallyrelative descriptors used herein interpreted accordingly. Likewise,descriptions of movement along and around various axes include variousspecial device positions and orientations.

The singular forms “a”, “an”, and “the” are intended to include theplural forms as well, unless the context indicates otherwise. The terms“comprises”, “comprising”, “includes”, and the like specify the presenceof stated features, steps, operations, elements, and/or components butdo not preclude the presence or addition of one or more other features,steps, operations, elements, components, and/or groups. Componentsdescribed as coupled may be electrically or mechanically directlycoupled, or they may be indirectly coupled via one or more intermediatecomponents.

The term “flexible” in association with a mechanical structure orcomponent should be broadly construed. In essence, it means thestructure or component can be bent without harm. For example, a flexiblemechanical structure may include a series of closely spaced componentsthat are similar to “vertebrae” in a snake-like arrangement. In such anarrangement, each component is a short link in a kinematic chain, andmovable mechanical constraints (e.g., pin hinge, cup and ball, and thelike) between each link may allow one (e.g., pitch) or two (e.g., pitchand yaw) degrees of freedom (DOF) of relative movement between thelinks. As another example, a flexible mechanical structure may becontinuous, such as a closed bendable tube (e.g., nitinol, polymer, andthe like) or other bendable piece (e.g., kerf-cut tube, helical coil,and the like). Accordingly, a short, flexible structure may serve as,and be modeled as, a single mechanical constraint (joint) providing oneor more DOFs between two links in a kinematic chain, even though thestructure itself may be a kinematic chain made of several coupled links.

While the memory in FIG. 1 is illustrated as a unified structure, thisshould not be interpreted as requiring that all memory is at the samephysical location. All or part of the memory can be in a differentphysical location than a processor. Memory refers to a volatile memory,a non-volatile memory, or any combination of the two.

A processor is coupled to a memory containing instructions executed bythe processor. This could be accomplished within a computer system, oralternatively via a connection to another computer via modems and analoglines, or digital interfaces and a digital carrier line.

Herein, a computer program product comprises a medium configured tostore computer readable code needed for any one or any combination ofthe operations described with respect to the shape informationprocessing module or in which computer readable code for any one or anycombination of operations described with respect to the shapeinformation processing module is stored. Some examples of computerprogram products are CD-ROM discs, DVD discs, flash memory, ROM cards,floppy discs, magnetic tapes, computer hard drives, servers on a networkand signals transmitted over a network representing computer readableprogram code. A non-transitory tangible computer program productcomprises a non-transitory tangible medium configured to store computerreadable instructions for any one of, or any combination of operationsdescribed with respect to the shape information processing module or inwhich computer readable instructions for any one of, or any combinationof operations described with respect to the shape information processingmodule are stored. Non-transitory tangible computer program products areCD-ROM discs, DVD discs, flash memory, ROM cards, floppy discs, magnetictapes, computer hard drives and other non-transitory physical storagemediums.

In view of this disclosure, instructions used in any one of, or anycombination of operations described with respect to the shapeinformation processing module can be implemented in a wide variety ofcomputer system configurations using an operating system and computerprogramming language of interest to the user.

All examples and illustrative references are non-limiting and should notbe used to limit the claims to specific implementations and embodimentsdescribed herein and their equivalents. The headings are solely forformatting and should not be used to limit the subject matter in anyway, because text under one heading may cross reference or apply to textunder one or more headings. Finally, in view of this disclosure,particular features described in relation to one aspect or embodimentmay be applied to other disclosed aspects or embodiments of theinvention, even though not specifically shown in the drawings ordescribed in the text.

1-20. (canceled)
 21. A system comprising: a teleoperated manipulator; amanually operated instrument; a position sensor coupled to the manuallyoperated instrument, the position sensor being configured to generatepose information for the manually operated instrument; and a controllerconfigured to: map a pose of the manipulator to a fixed world referenceframe based on three-dimensional pose information for the manipulatorknown internally within the system; and map a pose of the manuallyoperated instrument to the fixed world reference frame based on the poseinformation for the manually operated instrument.
 22. The system ofclaim 21, wherein the controller is further configured to map the poseof the manually operated instrument in the fixed world reference frameto a reference frame of the manipulator.
 23. The system of claim 22,wherein the controller is configured to map the pose of the manuallyoperated instrument in the fixed world reference frame to the referenceframe of the manipulator based on the three-dimensional pose informationfor the manipulator known internally within the system.
 24. The systemof claim 21, wherein the manipulator comprises an endoscopic cameramanipulator.
 25. The system of claim 21, wherein the manually operatedinstrument is coupled to the teleoperated manipulator by a tether. 26.The system of claim 21, wherein the controller is further configured tomeasure movement of the manually operated instrument by using the poseinformation for the manually operated instrument.
 27. The system ofclaim 26, wherein measuring the movement of the manually operatedinstrument includes measuring an insertion depth or an orientation ofthe manually operated instrument.
 28. The system of claim 27, whereinthe orientation of the manually operated instrument includes rotation ofthe manually operated instrument.
 29. The system of claim 21, whereinthe controller is further configured to: map a second pose of themanually operated instrument to the fixed world reference frame based onsecond pose information for the manually operated instrument generatedfrom the position sensor; and measure movement of the manually operatedinstrument by using the pose information and the second pose informationfor the manually operated instrument mapped to the fixed world referenceframe.
 30. The system of claim 21, further comprising a cannulacomprising a cannula opening at a proximal end of the cannula, whereinthe manually operated instrument is configured to be inserted into thecannula opening.
 31. The system of claim 21, further comprising areference fixture coupled to the manipulator, wherein the referencefixture moves with the manipulator.
 32. A method performed by acontroller of a teleoperated system, the method comprising: mapping apose of a teleoperated manipulator to a fixed world reference framebased on three-dimensional pose information for the manipulator; andmapping a pose of a manually operated instrument to the fixed worldreference frame based on pose information for the manually operatedinstrument generated from a position sensor coupled to the manuallyoperated instrument.
 33. The method of claim 32, further comprising:mapping the pose of the manually operated instrument in the fixed worldreference frame to a reference frame of the teleoperated manipulator.34. The method of claim 33, wherein the mapping of the pose of themanually operated instrument in the fixed world reference frame to thereference frame of the manipulator is based on the three-dimensionalpose information for the manipulator.
 35. The method of claim 32,further comprising: measuring movement of the manually operatedinstrument by using the pose information for the manually operatedinstrument.
 36. The method of claim 35, wherein measuring the movementof the manually operated instrument includes measuring an insertiondepth or an orientation of the manually operated instrument.
 37. Themethod of claim 36, wherein the orientation of the manually operatedinstrument includes rotation of the manually operated instrument. 38.The method of claim 36, wherein measuring the insertion depth of themanually operated instrument includes measuring the insertion depth asthe manually operated instrument is inserted through a cannula openingat a proximal end of a cannula.
 39. The method of claim 32, furthercomprising: mapping a second pose of the manually operated instrument tothe fixed world reference frame based on second pose information for themanually operated instrument generated from the position sensor; andmeasuring movement of the manually operated instrument by using the poseinformation and the second pose information for the manually operatedinstrument mapped to the fixed world reference frame.
 40. The method ofclaim 32, further comprising receiving the pose information for themanually operated instrument from a position sensor coupled to themanually operated instrument.